When OpenAI released its AI video-generation app, Sora, in September, it promised that "you are in control of your likeness end-to-end." The app allows users to include themselves and their friends in videos through a feature called "cameos"--the app scans a user's face and performs a liveness check, providing data to generate a video of the user and to authenticate their consent for friends to use their likeness on the app. But Reality Defender, a company specializing in identifying deepfakes, says it was able to bypass Sora's anti-impersonation safeguards within 24 hours. Platforms such as Sora give a "plausible sense of security," says Reality Defender CEO Ben Colman, despite the fact that "anybody can use completely off-the-shelf tools" to pass authentication as someone else. Reality Defender's researchers used publicly available footage of notable individuals, including CEOs and entertainers, from earnings calls and media interviews.

When Everything Is Fake, What's the Point of Social Media? Earlier this week, a heartwarming post about a girl, a puppy, and a police officer went viral across social media platforms. The post consisted of two dashcam images of a distraught 12-year-old who, desperate to heal her sick puppy, got behind the wheel for the first time and tried to drive to the vet. She was pulled over, but commended by a police officer for being "amazing, strong, compassionate, and smart," and the puppy was saved. Comments flooded in celebrating the bond between a girl and her furry best friend.

For instance, though a headline like "Red Sox Urged to Risk Passing on Alex Bregman in Favor of 427 Million Superstar" looks ordinary enough--and it seems, at first glance, to come from BBC Sports. But on closer inspection you may be on a knock-off called "BBCSportss," and the copy is lifted from Sports Illustrated. Elsewhere on that site you'll also find stories that aren't stolen directly from another writer, but instead read like a garbled remix of what other sports bloggers have written, and appear to be AI-generated. DoubleVerify, a software platform tracking online ads and media analytics, recently conducted an analysis of a collection of over 200 websites filled with a mixture of seemingly AI-generated content and snippets of news articles cribbed from actual media outlets. According to the analysis, these sites often chose their domain names and designed their websites to mimic those operated by established media brands, including ESPN, NBC, Fox, CBS, and the BBC.

Christopher Ren does a very good Elon Musk impression. Ren is a product manager at Reality Defender, a company that makes tools to combat AI disinformation. During a video call last week, I watched him use some viral GitHub code and a single photo to generate a simplistic deepfake of Elon Musk that maps onto his own face. This digital impersonation was to demonstrate how the startup's new AI detection tool could work. As Ren masqueraded as Musk on our video chat, still frames from the call were actively sent over to Reality Defender's custom model for analysis, and the company's widget on the screen alerted me to the fact that I was likely looking at an AI-generated deepfake and not the real Elon.

Audio deepfake detection is crucial to combat the malicious use of AI-synthesized speech. Among many efforts undertaken by the community, the ASVspoof challenge has become one of the benchmarks to evaluate the generalizability and robustness of detection models. In this paper, we present Reality Defender's submission to the ASVspoof5 challenge, highlighting a novel pretraining strategy which significantly improves generalizability while maintaining low computational cost during training. Our system SLIM learns the style-linguistics dependency embeddings from various types of bonafide speech using self-supervised contrastive learning. The learned embeddings help to discriminate spoof from bonafide speech by focusing on the relationship between the style and linguistics aspects. We evaluated our system on ASVspoof5, ASV2019, and In-the-wild. Our submission achieved minDCF of 0.1499 and EER of 5.5% on ASVspoof5 Track 1, and EER of 7.4% and 10.8% on ASV2019 and In-the-wild respectively.

In a crucial election year for the world, with the UK, US and France among the countries going to the polls, disinformation is swirling around social media. There is much concern about deepfakes, or artificial intelligence-generated images or audio of leading political figures designed to mislead voters, and whether they will affect results. They have not been a huge feature of the UK election so far, but there has been a steady supply of examples from around the world, including in the US where a presidential election looms. Here are the visual elements to look out for. In deepfake videos the area around the mouth can be the biggest giveaway.

There's a whole new way to get rich on the internet--at least according to a rush of YouTube tutorials touting the money to be made using AI to generate videos for kids. Searching for how to create kids content or channels on YouTube now pulls up tutorials offering roadmaps for creating simple animations in just a few hours. They advocate use of tools like ChatGPT, voice synthesis services ElevenLabs and Murf AI, and the generative AI features within Adobe Express to automate scripting as well as audio and video production. "IT'S NOT HARD," reads one of the top results' thumbnail images, while the title of another promises it is possible to generate a video with an original kids song "In Under 20 Minutes!" The virality fueled riches claimed to be on offer can be eye-popping.

When AI researcher Melanie Mitchell published Artificial Intelligence: A Guide for Thinking Humans in 2019, she set out to clarify AI's impact. A few years later, ChatGPT set off a new AI boom--with a side effect that caught her off guard. An AI-generated imitation of her book appeared on Amazon, in an apparent scheme to profit off her work. It looks like another example of the ecommerce giant's ongoing problem with a glut of low-quality AI-generated ebooks. Mitchell learned that searching Amazon for her book surfaced not only her own tome but also another ebook with the same title, published last September.

CEO Rob Meadows and co-founder Lars Buttler discuss the benefits and concerns surrounding artificial intelligence. Executives behind the American artificial intelligence (AI) company AI Foundation are cautioning against implementing kill switches in machine systems, arguing that such a move could increase the chances of a superintelligence that is hostile toward human civilization. According to a new Yale CEO Summit survey, 42% of polled CEOs agreed that AI could potentially end humanity within five to ten years. In citing the study, AI Foundation CMO and Chair Lars Buttler said the debate around AI needs to be elevated and suggested that people react emotionally to the new technology because of a lack of understanding about what is happening behind the scenes. However, both Buttler and CEO Rob Meadows warned of several concerns surrounding the advancement of AI and the possible creation of an artificial general intelligence (AGI) capable of reasoning and decision-making equal to or beyond that of a human. "With AI, you will always have this accidental danger, these accidental problems, you know?

A trickle of AI-fueled misinformation has turned into a powerful stream over the past year, with fake photos and videos--from Donald Trump's and Vladimir Putin's "arrest" to the Pope's "gangsta" outfit--highlighting the scope of the problem. "Deepfake" is an umbrella term for various types of synthetic content, created or altered with the aid of artificial intelligence, which can appear to show events, scenes or conversations that never happened. These types of creations come in a variety of visual, audial, and textual forms and can feature something innocuous, such as Jim Carrey in The Shining, or far more sinister and dangerous--like the fake videos of Joe Biden's "address to the nation," for example. Initially, deepfake technology was largely used to generate pranks and involuntary pornography. Now, it is increasingly deployed as a vehicle for misinformation--scientific, medical, financial, and, perhaps most worryingly, political. Newsweek previously reported on warnings that these technologies already present a real threat and have the potential to upend the democratic process in the 2024 election, with calls growing louder for regulators, big tech, and governments to intervene.